
Office de la propri§t6 

intellectuelle 

du Canada 

Un organisme 
d'lndustrie Canada 



Canadian 

inteiiectuai Property 
Office 

An Agency of 
Industry Canada 



-JAM 3 



Pon? 



Certification 






Certification 



La presente atteste^ue les documents[A;^^^^<^^l ist/ certify that the documents 



ci-ioints, dont la listeMgure ci-dessousf^^"^^ 
sont des copies authentigues-iaes|a<^Ur^/^^ 
ments deposes au Bureau^d^slbrw^^^ 




Specification ahil/Dfawirigs^as. OT for Patent Serial No: 

2,354,301 on Jul^lz^Opl', by^M^ ELECTRONIC SYSTEMS LTD., assignee 

of Christian Simon and Djamei Yahia Meltdah, for "Geometric Hashing for Model-Based 
Recognition of an Object".] 




O 



o 
»< 
o 

CD 

I 

o 



3) 

m 
o 
m 

< 
m 
o 





^^/^entrer^ncateur/Certifyin^pfficer 

October 23, 2OT1 



Canada 



0' 



.Date 



(ClPO 68) 
01-12-00 



O P I c 




CA. 02354301 2001-07-27 



-10442-20CAPR 



GEOMETRIC HASHII&G FOR MODEL-BASED RECOGNITION 



OF AN OBJECT 



CROSS -REFERENCE TO 



[0001] 



RE3 



lUWTED APPIiXCATZOMS 



Thia is the first application filed for the present 



invention. 



MXCROFICHB APPSNDIX 



[0002) 



Not Applicable 



TECHNICAL FIELD 



[0003] The prdsent 
and in particular to 
recognition of an obj 



BACKCROTTOD OF THg IW^WTIOW 



invention relates to machine vision, 
geometric hashing for model-based 
t. 



straight lines rema, 
preserved. Angles 



[0004] Technlcjues of visual object (6md/or pattern) 
recognition are increasingly important in automated 
manufacturing. biomedi::al engineering, cartography and many 
other fields, Model-biaed recognition techniques typically 
must solve the problerr of finding, in an image actjuired by 
a camera, an occurrence of a previously defined model that 
has been affected by affine transformation. Affine 
transformations may b« defined as treuisformations in which 



in straight 
how^vsr, may 



and parallelism is 
undergo changes and 



differential scale changes may be introduced. 

[0003] Images, which are the projection of a 
three-dimensional world onto a plane are dependant on the 
position, orientation and the intrinsic properties of the 
camera which is ac<iuiring the image. image distortions 
might be introduced by different scale factors in the X and 
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Y directions. Perspective distoirtionfi might be introduced 
due to the optical axis of the camera's lens being at an 
oblique angle to the object plane. Distortion might also 
be introduced by optical imperf dcitions of the camera's 
lens. Finally^ distortions might ajipear because the object 
is not seated on a planar surface. 

[0006] Known object recognition algorithms process 
accjuired images to find an occurrence of a match between an 
image and a model that is subject to affine transformation. 
When images are distorted (e.g., due to perspective, lens 
distortion, 6tc,) finding a match with the model requires, 
from the matching algorithm, more than affine 
transformation capability. 

[0007] Geometric hashing, as described in *^Geometric 
hashing: A generalised and Efficient Model-basdd 
Recognition Scheme" (Y. Lamdan and H. J. Wolfson, Second 
International Conference on Computer Vision, Dec 1988, 
pp 238-249), and "Affine Invariant Model-Based Object 
Recognition" {Y. Lamdarj, J, T. Schwartz, H. J, Wolfson, 
IEEE Transactions on Robotics and Automation, Vol. 6, 
No. 5, October 1990) has been proposed as a method of 
finding occui-rences between an image and a model with 
affine transformation and partial occlusion. 

[0008] In known geometric hashing methods , models of 
objects are represented : by interest points. For each pair 
of interest points, ai respective coordinate system is 
defined using the pair as a basis. The location o£ each of 
the other interest points can then be calculated within the 
respective COOr-dinate system, to produce a representation 
of the interest points t:hat are invariant to rotations and 
translations. For each coordinate system (basis), the 
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calculated coordinates of each int^est point is then used 
an index to reference a corraBponding bin of a hash 
table, into which . reference to the interest point and the 
b^sis is inserted- The fully populated haeh table is 
xntended to provide a representation of the model that is 
independent of rotation and scale effects, and contains 
sufficient information to enable a „»tch to be n«de even 
when an object is partially occluded. 

10009] As is well known in the art, object recognition 
conunences by acquiring an image of the object (e.g., using 
a gray-scale digital camera), and processing the image to 
detect points of interest, as with the model, each pair of 
interest points is ussd as a basis for a respective 
coordinate system, within which the locations of each of 
other interest points are calculated. These calculated 
coordinates are used to access corresponding bins of the 
hash table. if an accessed bin contains a reference to a 
model interest point (and basis) , then that model interest 
point (and basis) is accorded a vote. The basis that 
accumulates the largest significant number of votes is 
adopted as candidate, and extracted for further analysis. 

[0010J According to Lamdan and Wolf son ( -Geometric 
hashing: a generalized and Efficient Model-based 
Recognition Scheme-, supra), this geometric hashing 
technique can deal with partially occluded complex objects. 
However, in practice, geometric hashing often fails heavily 
occluded objects, or objects having sinple geometric shapes 
(e.g., triangles, rectangles, etc). This is because, in 
either of these cases, edge detection and analysis 
generally yields a sn^ll number of points of interest that 
pertain to the object in question. Thus if an image 
includes a heavily occluded object, or a partially occluded 
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simple object, the nuinbsr of detected points of interest 
may be too low to permit the correct record (model basis) 
to eccvmulate a significant number of votes. 

[0011] In addition, noise in an acquired image can produce 
errors in the computation of the coordinates of interest 
points, which may result in incorrect coordinate values 
being used to access the hash table. The problem of 
imprecision and computation errors can affect both points 
which define bases, and interest points that are used to 
vote. Since interest point coordinate values are a 
function of the chosen basis, errors due to iituprecision in 
basis point and in interest point locations are 
accumulated. These problems are a significant 

disadvantages of geometric hashing, and are discussed in 
"On tho Error Analysis of Geometric Hashing'* (Lamdan, H. j. 
Wolffion, Proceedings IEEE conference, Cort)puter Vision and 
Pattern Recognition, pages 22-27, 1991) and '*0n the 
Sensitivity of Geometric Hashing" (W, E. Crimson, D. P, 
Huttenlocher, Technical Report A. I. Memo 1250, Artificial 
intelligence Laboratory, Massachusetts institute of 
Technology^ 1990) . 

[0012] To avoid the above drawbacks, some improvements on 
geometric hashing have been proposed. In particular, 
instead of points of interest, the use of lines as af fine- 
invariant features to represent an object in an image has 
been suggested (See "A probabilistic Approach to Geometric 
Hashing using Line Features", Frank Chee-Da Tsai, Technical 
Report No. 640, Robotics Research Laboratory, Courant 
Institute of Mathematical Sciences, June 1993), In 
comparison to discrete points, lines provide a more robust 
representation of an object, because errors due to noise do 
not accumulate as rapidly as they do in calculating 
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cgordinates of discrete points . j According to Tsai, lines 
can be used as the basis of respective coordinate systems, 
and all COinpUtationg are made in lines space. 

[0013] While this approach yields a more robust 
recognition system for complex object, it is still unable 
to reliably detect heavily occluded objects, or simple 
objects with partial occlusion, 

[0014] Thd invention provides a method to improve 
geometric hashing and to solve the problems explained 
above - 

[001 B] 

SUMMARY py rag ItgygNTJON 
[0016] 

BRIEF DESCRIPTION OF T3gE DRAWINGS 

[0017] Further features and advantages of the present 
invention will become apparent from the following detailed 
description, taken in combination with th« appended 
drawings, in which: 

[0018] PIQ. 1 

[0019] FIG. 2 

[0020] It will be noted that throughout the appended 
drawings, like features are identified by like reference 
numerals. 
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DBTAIIiED DESCRIPTION pF THE PR3SFERRED EMBODIMENT 
[0021] The invention provides a geometric hashing 
technique for reliably finding one or more occurrences of a 
model in an acquired illlage 

[0022] model definitioh 

[0023] Step 1; Analyzfe image to detect interest points. 
These are preferably lotcal features of an image (e.g., edge 
points, edge discontinuities, etc.), but may also include 
locdti ons derived from image features (e.g. / a centroid) . 

[0024] Step 2: Approximate edg«fi with primitives. Each 
primitive is a line segment or curve segment that Is mapped 
through a sub-set of th« interest points. A ^best fit" may 
be used. However, preferably, a "less than best fit" 
approach Is used to ensure that continuity ie preserved. 
For example, a set of straight line segments can be used to 
approximate a continuously curved edge. If each line 
segment is a best- fit/ then adjacent line segments will 
frecjuently not intersect at their end points, so that 
continuity of the edge will be lost. Preferably, 
primitives are mapped .such that, for a continuous edge, 
adjacent primitives intersect at adjoining end points . 

[0025] In order to imtprove stability of recognition, bases 
are preferably selected: according to any one or more of the 
following rules: 

• use only highly '^stable" bases (i.e., those associated 
with comparatively lajrge primitives; 

■ allow a maximum of two bases for any one primitive; 

" distribute basis origins as evenly as possible over 
entire image. This !may include forcing a basis into a 
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region of an image; where features are indistinct if 
other portions of thb image have clear features favoring 
calculation of multiple bases; 

■ select basis orienta|:ions to approximately follow nearby 
primitives rather t^han angling sharply across image 
features . 

[0026] step 3: Represfsnt primitives es invariants. In 
general, this involves idefining a plurality of bases, and, 
for aach base, defining at each primitive relative to that 
base. In order to iitiprbve recognition stability, each base 
is preferably calculate?! from a relationship between two or 
more primitives. For ebcample the origin of a basis may be 
calculated as: an intersection point between lines 
extending from two pririiitives ; a centroid of two or more 
such intersections; or,; preferably, a centroid of two or 
more such intersections =wei9hted by the length at least one 
of the primitives participating in each intersection. 
Similarly, the orientation of a basis may be calculated as; 
parallel to a primitive ; near the origin; an average of the 
orientations of each ot the primitives used to calculate 
the origin; or a weightled average the of the orientations 
of each of the primitives used to calculate the origin, 
weighted by the lengths of each of the involved primitives. 

[00Z7] If desired, the dimensions of the invariant 
representations of the ' primitives can be normalized to 
render the model scale independent. However, this 
increases the size of tlhe hash table and complicates the 
problem of error tolerance. Accordingly, in preferred 
embodiments dimensions are not normalized and therefore 
absolute dimensions are' mapped to the hash table. This 
renders the model scale ! dependent. However, scale can be 
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handled by any of a nu«\ber of approaches including: 
multiple hash tables (one hash table per scale step) ; 
adjusting the size and/or number of error bars to guarantee 
that at least one scsale step will result in a target 
primitive intersecting -an error bar; or populating the hash 
table with references i for model primitives in each of a 
plurality of scale atepa, in which case each reference in 
the hash table is of the form (basis, scale, primitive) . 

[0028] Step 4: Sample primitives with an arbitrary 
granularity. The granularity chosen is based on a balance 
between speed and robustness. In general, a finer 
granularity (i.e. mord samples) requires more processing 
time but yields improved tolerance to occlusion. 

[0029] Step 5: for eadh Saxrple: 

[0030] Step 5a: Map saanple coordinate to an associated bin 
in a hash table. 

[0031] Step 5b: Insert a reference to tbasiB, primitive) 
in the bin, and into one or more adjacent bins to create 
"error bars", the number adjacent bins used, and their 
orientation relative to the "main" bin are selected based 
on an expected image resolution error of the image 
processing system. in particular, based on the resolution 
of a camera used to acjquire an image, the location of any 
point (or an edge) can only be determined within some 
error. Accordingly, if a target primitive lies within a 
predetermined error zpne (or corridor) around a model 
primitive, then the target primitive should be considered 
to lie on the model primitive. The size of the error bars 
are selected to ensure that this occurs. 



[0032] Step 6: Save haish table. 
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During Run Time 

[00331 Step Rl: Analyze target image to detect interest 
points. These are prefierably locel features of an imase 
(e.g., «dge points, ed^e discontinuities, etc.). but may 
also include locations d»rived from image features (e.g., a 
centroid) . 

[0034] Step R2: Approximate edges with target primitives. 
Each target primitive iis a line segment or curve segment 
that is mapped through a sub-set of the interest points, in 
the saime manner as descsitoed above. 

[0035] Step R3: Represent target primitives as invariants, 
in the same manner as described above. 

[0038] Step R4; Find end-points of target primitives and 
locate corresponding bins in the hash table. 

[0037] Step R5: Locate bins crossed by an imaginary line 
extending between the ■ "end-point bins'. Any method of 
representing a line as discrete elements (e.g., pixels) may 
be used for this purpose. For example, a conventional 
Bresenham method, which is normally used to identify the 
pixels of a screen thati need to be illuminated in order to 
display a line. in thie present invention this method can 
conveniently be used to identify hash table bins instead of 
piisels . 

100381 Step r6 : If any located bin includes a reference to 
a (basis, model primitive), then: 

[0039] Step R6a: Caltpulate the length of an orthogonal 
projection of the target primitive onto the referenced 
model primitive. TWLs length is a region of overlap 
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between the target and model, lying within the error 

t 

corridor about th« model primitive, as discussed above. 

[0040] Step R6b: Accuiikulate the calculated length in an 
'"bucket" associated with the identified basis. The 
accumulated length stored in the bucket may be referred to 
as the ^'target coverage!'. 

[0041] Step K7: Repeat" steps R4 through R6 for each target 
primitive within the target image. 

[0042] Step R8: The tital length of the model primitives 
(for any one basis) ton be referred to as the * total 
coverage'" . The ratio pf the accumulated coverage for any 
one basis to the "totjal coverage* of the model is the 
^relative coverage" o£« that basis. The basis with th* 
highest relative coverage is selected as a candidate for 
further analysis and verification of a match in between the 
basis and the model . 

[0043] The «mbodiment;(s) of the invention described above 

is {are) intended to bei exemplary only. The scope of the 
invention ia therefore iintended to be limited solely by the 
scope of the appended qlaims. 
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WE CUUM; 

1 . A raethod of matthing a pattern to a predetermined 
model repreaentdd by a hash table, the method 
comprising steps !of : 

analyzing th® patitern to identify interest points; 

processing the ixjterost points to define one or more 
primitives apjproximating edges of the pattern; 

representing the! target primitives as invariants 
relative to a| selected basis; 

finding respGcti\^e end-point bins of the hash table 
corresponding: to each end-point of each target 
primitive; ; 

searching the hajsh table between each of the end- 
point bins tio identifying a bin containing a 
reference to ^ (basis, primitive) of the model; 

calculating an otthogonal projection of the target 
primitive Onto the referenced model primitive; 
and 

accumulating thei calculation result in a bucket 
associated with the referenced basis. 

2. A method of poplulating a hash table, the method 
comprising steps of: 

analyzing a modeli image to identify interest points; 

processing the interest points to define one or more 
primitives approximating edges of the model 
image ; 

representing the: model primitives as invariants 
relative to a • selected basis; 



CA 023S4301 2001-07-27 



9-10442-20CAPR 

: - 12 - 

sampling each mlpdel primitive at a predetermined 
granularity; • 

mapping the locktion of each sample point to a 
corresponding! bin of the hash table; 

inserting an reference to the (basis, primitive) of 
the model into the bin; and 

inserting the rejference into two or more adjacent 
bins of the h^sh table. 

3. A method of defining a basis of local coordinate 
system of a plurality of primitives approximating an 
image, the method' comprising steps of: 

calculating an origin of the local coordinate system 
based on a selected relationship between two or 
more primitives; and 

calculating an orientation of the local coordinate 
system based on an orientation of at least one of 
the two or more primitives . 
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